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ABSTBACT 

Beported are the results of instructional development 
projects at Utah State University, funded under mini grants, faculty 
development: grants, cr de.velopmental grants to departments. These 
projects involve rede^sign of courses in media production, library 
resources, pattern design and fitting, counselling psychology, 
guantitative methods, sociology, reading, accounting, fluid 
mechanics, materials science, soil mechanics, and wildlife science. 
Individualized approach and media presentation are used in most 
courses. Comparative data of the alternative instructional method and 
original methcd, and student feedback are included in this report. It 
concludes that despite limitations in evaluation design, substantial 
results in both studc^nt achievement and student attitude have been 
demonstrated, and faculty involvement in various projects is 
meaningful. (Authcr/SC) 
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ABSTRACT 



Nationally, attempts to measure impact of instructional de- 
velopment (I.D.) on student achievement and attitudes are rare 

Tl\is report sets out to document the impact of I.D. efforts 
at U.S.U. through »:pecific results from individual projects funded 
under mini grants, faculty development grants, or departmental de- 
velopment grants 

A limitation of these studies is the lack of complete control 
over intervening variables inherent in research with human subjects. 
None of the studies are flawlesss from an experimental standpoint. 
However, each attempts to provide valid data through the most 
feasible means available 

These studies have generally involved faculty in the planning 
and conducting of the studies. Faculty involvement is felt to 
stimulate the use of the study's findings,' a major advantage of this 
approach 

Minigrants (up to $500) are given to encourage modest projects 
initiated by faculty members. While evaluations of these are gener- 
ally sketchy, in three cases, extensive evaluation has been conducted. • 

(1) IM 551: Production of Medi a Materials . The lecture demon- 
stration method was compared with three variations of 
instructional television (ITV) presentations. The only 
significant difference noted on several measures of 
attitude and achievement was found in the significantly 
lower attitudes expressed in one fairly unstructured 

class using the experimental ITV approach 

(2) History 300: Sources and Literature of History and 
the Social Sciences . This minigrant sought to infuse 
interest into a class requiring extensive work with 
library resources. A measure of student attitudes 
upon completion of the course showed partial success 
in overcoming the stereotype of a boring class. Stu- 
dents saw the self-instructional modules as a sig- 
nificant improvement 

(3) HECE 260: Pattern Design and Fitting . Two modules 
involving slide/tape presentations and clothing fitting 
experiences were compared with the previous method of 
demonstration instruction. Students performed equally 
well unr^er the new approach, reaching the required 
criterion level, and expressed sentiments favorable 

to this method 



Faculty (.lovolopmeiit ftriuits, >;iMKMMlly involving the released 
time of the rocipieat. allow t'or more extensive development work and 
more tliorou^»h evaluation efforts than is feasible for minigrants M 

(IJ CoiuiselinK Psychology: Observational and Record! n^> 
^Skil Is in H ehavior Modi fi cat ion , This approach using 
slides, cassette tape and a' workbook was df/.onstrated 
to be effective in presenting material; howovMr, it 
was not compared with any other existing; n\et},od. Stu- 
dent feedback characterized the approach as interesting, 
valuable, and efficient 14 

(2j B.A. ()01: Quantitativ e Methods for MBA Students , These 
modules were developed for self-instruction in prerequi- 
site skills for MBA students. The information gathered 
to date indicates (1) the completion rate of those who 
register and take at least one e: i is higher for on 
campus students (63,2%) than for those off campus (58.1%). 
Achievement under tlio modular approach has not been com- 
pared formally with the classroom approach 15 

(3) Soc. 101: Introduction to Sociology . The revised lab 
sessions, conducted by graduate students, showed consider- 
able variation depending upon the graduate assistant. 
Student opinion regarding the lab sessions was tracked 
over three quarters but did not indicate major impact 

from this project 17 

(4) Sec. Ed. 610: Improvement of Reading in Secondary 
Schools . As a prelude to course changes, two studies 
of reading were conducted. The results suggested that 
(1) Utah students read considerably less on an indepen- 
dent basis during high school than during junior high 
years, and (2) experts in the field see the teaching 
of skills in study skills and reading comprehension in- 
struction as priority items for secondary teachers. Two 
self-study modules were developed as a supplementary, 
alternative approach to this material for teachers. 
While both modules were found to teach the material to a 
specified criterion level, only under the first were 
students found to perform as well as the traditional 
lecture approach. Student achievement under the second 
module was found to be significantly lover than the 

previous method, indicating a need for revision 19 

(5) Acc. 561: Accounting Theory . Both a minigrant and a 
faculty development grant were involved in this project 
to integrate current pronouncements of accounting stan- 
dards into existing curriculum. Four developments were 
noted: a statewide survey of accounting teachers 

found widespread disregard for these pronouncements, leading 
to (2) a nationwide study of members of the association 
which found widespread acceptance for the integration 
of this knowledge into the existing curriculum rather than 
creating new courses to Dach it. (3) An increasing 
awareness of this information has been found among 
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inconunv; siiulontj;. i i f;i t i iij.; v.roater atrention bein^ 
paiti hy other pro Tt'ssors in the department to these 
pronouncements; anil (4) the accounting theory course 
has heen redesigned and demonstrated more effective 
in the final version than in the first draft stage 21 

One major thrust of Instructional Development efforts has been 
involvement at the departmental level. The two departments involved 
in these efforts thus far have heen Civil lingineering and Wildlife 
Science 25 

(IJ CM .S.Si/. I- hi id Me chanics . This course was developed 

usuig a self-paced, modularized approach as one of the 

f'rst faculty development grants given. When compared 

with students in a lecture presentation, students 

under this method performed equally well on a test of 

problem solving skills (open book test) and performed 

significantly better on a test of concept learning 

(closed book test) 25 

(2J MI: 415: Materials Science . Twenty-five self- instruc- 
tional slide tapes were developed as a basis for this 
class. On comparisons of student achievement and 
attitude, the revised course demonstrated superiority 
over the previous lecture method 28 

(3) CL: 4.;/: Soil Mechanic s. With the cooperative efforts 
of thrwo professors in this field, a new textbook in 
Soil Median ics was developed. Supplemented with a 
series of overhead transparencies and a revised course 
sequence, the new course gets high marks for instructs 
enthusiasm and student attitude. The finding of sig- 
nificant jump in enrollment over previous years in the 
next Class in this sequence (an elective) has been in- 
terpreted as an indicator of higher student interest il 

( 4) WS (iame Ri nls arid^ Mamma J s . Developed by a profes- 
sor and two graduate students, this individualized in- 
structional bookhn was supplemented with color slides 
to teach three weeks of the course material. A compari- 
son of mid-teriii resn 1 1 s, admi 1 1 od 1 y over different 
material, hut witfi rx.ims oi' ct(iial difficulty suggests that 
students avhievc at least as woi 1 using the new format. 
Student ••pinion tov.ird tlie :i jijjr(\-ioh was favorable 7*2 
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l-ORIiWORI) 

Instructional development at Utah State University has attempted to 
engage faculty members in fairly modest development projects with maximum 
involvement. What has been accomplished thus far? To what extent are the 
changes documented? 

Evaluation studies in the field of instructional development are rare 
at present. Gaff (1975) notes that -promoters of instructional improvement 
programs have been too busy getting things in motion to worry about evalua- 
ting what they are doing" (p. 161). To some extent, this observation is 
applicable to USU; however, in the past two years, with funding from the 
Fund for the Improvement of Post Secondary Education (FIPSE) considerable 
effort has gone into providing evaluative data for program decisions. 

A previous report entitled "ID Under the Microscope" dealt with fa- 
culty perceptions of the instructional development program. This report 
deals with impact upon students, specifically with changes in attitude or 
achievement as a result of I.D. involvement. Not all projects have been 
of the type to allow such documentation, but where available, such data 
have been reported. 

Appreciation is expressed to the faculty, students, and staff who 
assisted in the gathering or processing of information. 



J. N. Eastmond, Jr. 
Kathleen L. Van Horn 
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1 . rNIKODDCTION 

The goal ot instructional development from the standpoint of the center 
at Utah State University is one of continuing insitutional renewal. The 
outcome of such renewal can be assessed in a niunber of areas. This study 
cites evidence in two such areas: (1) in student achievement, and (2) in 
student attitade toward the learning environment. Measurement in the 
two areas is certainly interrelated, although the relationship is not always 
direct and simple. For example, if improved instruction boosts achievement 
the student's attitude toward the course may improve. Conversely, poor 
instruction can frustrate the learner and lead to negative attitudes to- 
ward the course and toward his own abilities. However, u number of faculty 
members have expressed misgivings about viewing student attitude alone. First 
because of concern that a more demanding class may not be viewed favorably 
by many students, and secondly, because appreciation for a course may change 
over time. None of the studies reported herein attempts to document 
attitude toward a course longitudinally, over a period of time, although 
such data collection is planned for future studies. Whenever possible, how- 
ever, an attempt has been made to examine both attitude and achievement 
result:^> and thereby obtain a more valid composite picture of the course. 

In conducting these studies, an pttempt is made to divorce the evalua- 
tion from considerations of teacher personality and style to focus upon more 
impersonal factors such as course organization, content, and instructional 
technology employed. This approach suits both the style of the USU center, 
which d^als more with ^'instructional products rather than teaching skills" 
(Eastmond, 1975c, p. 2Q) , and also helps to make the studies less ego- threatening 
to the faculty menber. To be sure, faculty members are bound to be ego- 

9 



involved with the courso they holpcnl crcatt^ but by oxaminiiiK the cowrsv 
« 

from an imporsonal viowpoiiu, loss dot'tMis i voncss and more i-onst rue t i vo 
outcomes can be fostered. 

•Hiis study emphasizes numerical scores or "hard data'* in both achieve- 
meat and attitud nal analysis, recOKiiir^inK that a portion of the reality 
of the teaching/learning process is not captured by this approach. While 
anectdotal and hisoric .ypes of data arc kept, this report focuses on 
an interpretation of quantifiable data. Thus, the role of the evaluation 
reported herein is to systematically collect and process varieties of data 
to document the impact of instnict ional change. . 

The aim in conducting such a study is to involve faculty members exten- 
sively in both the I.D. process and in the evaluation of their revised instruc- 
tioi\aJ product. Consequently, evaluation data are collected in diverse ways 
to meet the professor's needs and desires. If he is interested in assessing 
the attitudes of his students toward new methods incorporated into a course, 
we work with him to clrvelop the necessary measures. The instrument used asks 
whatever the professor wishes to know presumably providing both the formative 
and siunmative evaluation information. Typically, the ID office will admin- 
ister the measure, analyze the data, and help interpret the results with 
the faculty members involved. If warranted, changes can be made in whatever 
"product'* or "teaching method" is being assessed. 

While some limited evaluation is prssii le with the minigrant program 
(funded at less than $500), faculty developia. .\t grants involving faculty 
released time offer the chance to use more sophisticated designs in col- 
lecting evaluation data. After the planning stage is completed, it is 
-ossible to continually evaluate progress toward the original objectives. 
The student and/or faculty questionnaire to assess attitude is again approp- 
riate. But, furthermore, quasi-experimental research designs allow assess- 
ment of changes in student achievement. Such designs are considered quasi- 

10 



fxprrinicru.i 1 (Csinihi^l 1 atul St.inlcy, \\H>^] siiu'i.^ the saiiipit* is fjiMUMMlly 
i-omposivi i)f the cxistiiif: ^M.'iss ratluM- Than a raihlomly chosm f;roup. Idrallv, 
the stops Ml imploitUMil in>; one such ilesi^n wouhl he; (I) Dtirin^: first 
or planning: (luarter of a fncolly (levelo[)menl K'^'^*^^ ► wliile the professor 
teaehes the eourse usin}: the I rati i t iona I approach, haseline stiulent attitmle 
and achievement are measured. (2) During: the second (juarter the professor 
ilcvelops the new course. UopefuUy. his new course plan would include 
plans for all measures to he comparable to the ori^jinal measures, Duriup. 
the third quarter, while teaching; the new course, attitude and achicvemerit 
of skills are .'t^;ain ineasureil. Some means of equating the original groups-- 
either through a pretest measure or entering GPA—is desirable. 

As will be apparent in the spjcific case studies that follow, a variety 
oL' :;xperimental designs and cvaluat on instruments have been used. From 
a practical standpoint, extensive j r»vol vemcnt in evaluation research de, .:ids 
upon initiative for the evaluation coming from the faculty member. Without 
his support, the effort will likely be V' ;wed as an impoL:ition and the 
res»ilts will often be ignored. 

TTie same philosphy guides both the ID process and the evaluation efforts 
at USU, A faculty member will find a process useful when he part icipat .:s 
and contributes his own ueas. lie will find the process useless or imposing 
if he docs not. This finding is true of whole departments. The limitations 
and problems involved in collecting ^'hard data" stem from such an operating 
philosophy. Many times a professor who has developed a new course or 
changed only part of a course is reluctant to assess either student attitude 
or achievement. The course is his property; he may fear the outcome of 
evaluation with its possible negative implications. Course redevelopment 
involves not only changed teaching methods, but sometimes deleted or changed 
content. Rxams for the new course may assess more difficult skills. If 
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thf v-oiir-..- < uiit.'iit . li.miM--. .Ir.im.it ic.i I ly. v-.. I 1 .h t i .om;).!!-.!!* U> i. lu cvcmcMt 
a.ita on .1 liiMl ,.x..,ti virfii.illy iinpo.;-. il, 1 nivni pr.-.vi,r resource votiM raint 

llu:r«jforo. wo fitul wi must employ wh.ilovor t<'olini(iiics .iro oC im)st ir.c to 
accomplish a methodolonir.il ly soiiiul ai\J accoptablo evaluation for the faeiilty 
member. IT such an effort appears threiiteniii>: to the eliciit, we purposely 
hack of rather than ahort tnc 'iMS'-ioii. The following; section ilesorihes 
piojects where such data were collectable ami couhl be .summar i :.eil . [k)th 
positive and nonative findin};s are discussed, bul a word of caution is in 
oriicr. 

Data are used in social science research to "provide explanations." 
However, the necessity for the researcher to interpret the data and to draw 
conclusions should not be forgotten. Keality is extremely complex, the 
effects of various factors and their interactions arc manifest in ways far 
too complex to be understood vith the rather basic tools--jtatistical and 
othcrwisc--cmi.loycd in the relatively simple research designs described 
below, -nie point is made simply to put these techniques into perspective. 
Moderate attempts at controlling intervening variables have been made. How- 
ever, such attempts generally represent relatively rough curs at describing 
a project's effectiveness. 

While the limitations are evident, so too are the strengths. Each of 
these projects was evaluated with the cooperative effort of the faculty 
member developing them uid the ID evaluation staff. The results from these 
measures do not go iinheeled: clients generally take a hard look at the 
results from their projec-.s and make needed revisions. 

Projects are described individually and results reported, both positive 
and negative findings. Where possible, the most important factors involved 
are mentioned with the aim of explaining the observed phenoirena. Projects 
proceed from the simpler to vhe more complex, from mini grants to faculty 
development grants to departmental development work. 

Er|c 12 



II. RESULTS OF SELECTIVE MINI GRANTS 
Following the building of awareness of the Instructional Development 
Center's existence on campus, small grants ranging from $200 to s^SOO are 
offered to all ::aculty members. Called "Mini Grants*', these relatively 
small outlays of money are useful in encouraging innovation among faculty 
members, to try out ideas the person has always wanted to do. In higher 
education, tho grant has specific social value. A professor who receives 
a grant achieves an immediate status reward (DeBloois and Alder, 1973). 
At the same time he must commit himself to the output required to ful- 
fill the grant requirements. Each mini grant asks that the recipient invest 
his time in designing learning strategies to improve instruction, i.e. for 
part or all of a course. Included, to a greater or lesser degree depending 
on the project, are I.D. consultation and guidance on problems such as 
media production, testing, and evaluation. 

It has been recognized that, given the limited resource commitment, 
no extensive evaluation effort is generally warranted. The three projects 
described below represent exceptions in that it was important to the 
client to obtain results regarding the project. Keep in mind, however, 
that over one hundred such projects have now been completed and that the 
three reported below are somewhat atypical of the majority, i.e. considerably 
more data were gathered. 

Instructional Media 551: 
Production of Instructional Materials 

During winter quarter, 1975, the Instructional Media Department at Utah 
State University received a mini grant for $500, awarded through the Division 
of Instructional Development. The purpose for the project was to develop 



a series of 24 video tapes for course IM SSI. Particularly in.jsortant for 
this class was the demonstration of fairly elaborate techniques which had 
to be set up and taken down for each class previously. The aim was to save 
instructor time during demonstration which could then be used for indi- 
vidual lab work with students. 

An extensive evaluation of alternative methods of presenting the 
course was conducted with three sections of the course during spring quar- 
ter, 197S, and then an additional follow up trial during fall quarter. The 
three sections were structured as follows: 

GroupJ_: Beutler's Experimental: Students had no scheduled class 
time, but instead, viewed video taped class presentations 
at their convenience. While a regular lab period was 
scheduled especially for this class, additional evening 
lab sessions were available. The regular course instruc- 
tor was used to conduct the sectiori. 
Grou£_2: Beutler's control. Students received a class lecture/demon- 
stration but did not view the video presentation. A lab 
period was scheduled. A regular course instructor conducted 
the section. 

Groi2_3: Hambelton's Experimental. Students saw the video taped pre- 
sentation as a class and then proceeded to a lab period. 
The option of making up a missed presentation was available 
during the ""rllowing class's lab session. An advanced 
graduate student teaching assistant conducted the sessions. 
Group 4 : Smellie's Experimental. Students viewed the video content pre- 
sentations prior to class and were required to attend lab 
sessions. The instructor was available during this time 
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for questions or assistance. Projects were required 
to be turned in on pre-set deadlines, i,e, a definite 
pacing system for student work wf^s employed. 
The experimental design employed in this research is called a '*Non- 
equivalent Control Group,'* as described by Campbell and Stanley (1966), 
Basically, the design used three experimental groups and one control 
group made up from pre-existing groups, in this case students who 
signed up for the course at a particular hour (9:30 or 11:30 a.m,), 

While this procedure does not employ randomization in assigning 
subjects to groups, a random method was employed to designate experimental 
and control groups. 

Based upon the minimal differences between groups on a pretest 
of knowledge of course content, it was felt that groups could be con- 
sidered equated on entering knowledge. Therefore, a simple one-way 
analysis of variance statistical comparison was used. The independent 
variable for various ANCOVA comparisons were the scores on particular 
tests (pre-test or post-test) and total accumulated points for the 
course. Results of the pretest and post-test of course content given 
to groups 1-3 demonstrated no significant differences between the 
groups , 

Twelve projects were required for this course from every student. 
The total of these points was used to assign a student's grade. It 
should be noted that scoring was done in reverse order, i,e, better 
performance is shown by a lower score. Table 1 shows these results for 
the three groups. 
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TABLE 1: 

Comparisons of Total Project Scores, IM 551 

Beutler's Eeutler's Hambleton's Smellie's 

Experimental Con trol Experimental Experimental 

X 187.73 172.28 172.25 181.70 

S.D. 31.56 43.86 28.03 30.57 

n 11 18 12 20 

There are no significant differences across groups on this measure 
(F = .633; df 3, 57), 



For the measure of attitudes, no pre-test of attitude was administered, 
as the questions asked are clearly appropriate only after some exposure to 
the course. The findings of the Analysis .-riance for these scores, un- 
like the previous three measures, do show sigi.ificant differences (F=14,05, 
df). Table 2 summarizes the observed differences. A comparison between 
means shows all groups to be significantly higher than Beutler's Experi- 
mental section: 



TABLE 2: 

Comparison of Attitudinal Meausures, IM 551 

Beutler's Beutler's Hambleton's Smellie's 

Experimental Control Experimental Experimental 

^ 56.08 74.88 70.23 74.95 

SD. 11.70 8.06 10.10 6.48 > 

" 13 17 13 19 

None of the other sections differ significantly, i.e. the differences could 
just as well be due to chance, (F=14,05, df 3, 58) 



Examining the specific items on the attitude measure, it is apparent, " 
that students were uncomfortable with "getting motivated to do what had to 

Er|c 16 



9 



be done" and in the lack of personal contact with the instructor. These 
findings were corroborated by the number of students who dropped out 
after the first class period and by the number of student.s who took grades 
of '^Incomplete." In both cases, the numbers were more than double that 
of the other classes. 

From these findings, it was not possible to conclude that either the 
lecture or the videotape approach was more effective than the other. In 
none of the four tryout cases was achievement significantly different. In 
only one of these did student attitudes differ significantly. These results 
clearly indicate a need for classroom structure, packing, and individual 
contact with the instructor. At this point, any differences in effectiveness 
between the two approaches, video vs. lecture, are not apparent. 

. History 500: 

The Sources and Literature of History and the Social Sciences 

This course was intended to prepare a student with library informa- 
tion retrieval skills for graduate research or the teaching of history. 
The prevailing attitude toward the class, held by both students and fac- 
ulty, had tended to label it as a necessary evil, a required set of library 
skills which could only be obtained through painful and laborious 
experience. One aim of Dr. Alder's course revision, using a mini grant 
and staff support from the ID office, was to use a set of novel techniques 
to get the required message across and thereby upgrade the image of the 
course. A second aim was to individualize information dissemination to 
thereby allow class time for group activities. Miile the time open in 
class has been decreased, the student advisement hours have reportedly 
increased considerably. 

Additions to the course included a 70 page workbook, an audio-tape 
library tour, ten slide-tape modules, a filmstrip and one computer assisted 
^ instruction lesson. All but the last two of these were developed during 

ERIC 17 
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the mini grant. Students were assigned to work independently, turning in 
an assignment at the completion of each unit, with one formal lass meeting 
per week and the instructor available for individual advisement. 

The revisions were made during the fall and winter quarter, 1975-76, 
with th<* course iriiplemented during spring quarter. A survey of attitudes 
toward the course was administered at the end of the quarter. No 
measure of achievement was attempted. 

Strong points identified about the course were (1) that the modules 
were a significant addition to the course; (2) that formal meetings once 
per week were adequate, given the self-paced modules; and (3) that students 
likely would not have learned the material as well without the effort of 
the performance-based t.>st. On the semantic differential portion of the 
questionnaire, the course was characterized as being "necessary," "good", 
"useful''. It was also seen as "unusual", "imaginative", and "up-to-date". 

Some points made in the questionnaire suggest that much of the stereo- 
type of the course was only partially overcome. For example, there were 
mixed feelings about the time required for the library tour being exces- 
sive and the amount of repetition of course material. The same kinds of 
mixed reactions were apparent for the semantic differential, with virtually 
a neutral mean position for "Boring vs. Interesting", "Meaningful vs. Busy- 
work" and "Unpleasant vs. Enjoyable". Apparently, the stereotype still 
holds to some extent, at least for certain students. 

Some additional points suggest possible areas for course revision. 
Students noted uncertainty at times about what was required. Seven stu- 
dents noted equipment malfunctions with the recorder during their tour, 
while others were reasonably satisfied here. Future revisions are anti- 
cipated to strengthen the modules; however, these will likely not be 
extensive in the near future. 

er|c 1^ 



Home Economics and Consumer Education 260 : 
Pattern Design and Fitting 

During the 1975-76 school year a mini grant was awarded to Jean Alder 
of the Department of Home Economics and Consumer Education to develop por- 
tions of a course in Pattern Design and Fitting (HECE 260). Annette Fraser, 
^ a masters degree candidate, using the R§D approach, worked closely with 
the ID staff to develop two modules for this course, this effort forming 
the basis for her masters degree work. 
The modules were as follow^: 

1. ''Five Factors of Fit/' consisting of a 17 minute slide-tape 
presentation dealing with the principles and standards of 
clothing fit. Designed for use on a self- instructional basis, 
it can also be used in a group setting. 

2. "Fitting Phenomena,'* is a seven minute slide tape presentation 
and is completely self-paced. This module incorporates student 
participation in the fitting of a standard garment on a half- 
scale replica of a deviant figure. Time required for comple- 
tion varies with the student, generally from 35 to 75 minutes. 

Evaluation of the modules was done with 15 people who had already 
taken the course in previous quarters, sometime within the previous nine 
quarters. The average time since course completion was 3.8 quarters for 
this group. The average age of this group was 21.8 years ar the time of 
course evaluation. 

These people were contacted by phone and asked to participate in an 
experiment. They were scheduled to work through the modules in groups of 
three. Appropriate questions selected from the fitting test used in the 
HECE 260 class were used as a Pretest and Post-test. The individual 
test papers--kept on file by the department--were rescored to determine 
how well the subject had performed in that content area when they origin- 
ally took the course. Then randomly, the pertinent questions were divided 
up into the pretest and post-test, each weighted to have an indentical 
maximum score with the original portion of the final test. Thus, a com- 
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parison of achievement war possible at three points in time: (1) when 
students originally took the course; (2) at pre-test time, with no review; 
and (3) following module completion. In addition, students were evaluated 
on module success by a criterion based (mastery) test wnich required that 
100% of the students reach 80% of the objectives. All students met this 
goal. Statistical information was used to back up the study. Participants 
were also asked to complete a short questionnaire dealing with feelings 
toward the module. The time required to complete the second module was 
recorded (average: 55 minutes), but no comparison with the previous time 
required was possible. 

Results 

On the te5t of achievement, subjects averagsd highest on the original 
class test (23.87), dropped considerably over time as measured by the pre- 
test (19.00), but improved considerably for the post-test (22.27). Sta- 
tistically, the pre-test results v/ere significantly different (p<.05 level) 
from the other two tests, but the final test and the experimental post-test 
did not differ significantly. From this finding, one can coiclude that the 
module teaches course content to a level equivalent to that previously 
attained in ^he class. While one would expect previously learned material 
to be learned somewhat more rapidly (Travers, 1972), situational factors 
were probably more favorable in the previous classroom environment and thus 
may balance out the relearnin;? aspect. 

Results on the opinion measure were favorable toward the module. Par- 
ticularly for the thiee questions dealing with module desirability, responses 
were highly favorable, averaging 4.7 on a five point scale. Specifically, 
these questions assessed attributes toward (1) the option of repeating the 
content presentation; (2) the convenience factor of individual use in the 
classroom; and (3) whether these modules would be rated as a ^'worthwhile 
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addition" to tn** class. Thus, while resulting in equivalent achievement 
on a test of couise content, these two modules were viewed favorably by 
students and represent, at least from a learner's point of view, a sig- 
nificant and worthwhile addition to the course. The evaluation 
design, using a retrospective analysis of student test scores, is unique 
aracng the sot of evaluations reported herein and is quite ingeniously con- 
vincing. From the point of view of the ID Center, this piojecr represented 
the suc':essful use of a capable graduate student as an intermediary be- 
tween ID sta:^f and the HECE faculty member. 
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III. RESULTS OF DEVELOPMENT GRANTS TO INDIVIDUAL FACULTY 

Faculty Development Grants involve an important change in emphasis 
from that of minigrants. Besides designing learning r^trategies for a 
course, the professor must become personally involved in the instructional 
development process. Competitive Faculty Development Grants of $1,5000 to 
$4,500 are offered to accomplish this. A Faculty Development Grant re- 
quires that a professor spend one quarter of his/her own time in planning 
a total course with the Instructional Development personnel. The planning 
quarter is followed Hy a quarter of released time, during vfhich I.D. per- 
sonnel assist the faculty member in the actual development of materials. 
Finally, during the third quarter, which should be the concluding quarter, 
the redesigned course materials are tested and evaluated as they are used 
in the instructor's classroom. However, due to personal needs, the pattern 
for carrying out a Faculty Development Grant may vary. For example, a num- 
ber of recent grants ha involved graduate students as well as faculty 
members in carrying out the work. Generally, during the released-time quarter, 
the department involved provides half of the professor's salary while 
the Instructional Development Division provides the other half. Since other 
professors must accept responsibility for the grant recipients classes 
for this time, it is important that the whole department initiate and 
support the change effort. 

Counseling Psycholog y: 
Observational and Recording Skills in Behavior Modification 

Partial funding by the Division of Instructional Development formed the 
basis for a multimedia program for teaching observational and recording skills. 
Two doctoral candidates, Gerald D. Hecker and Darlene L. Adams, Carried out 
the development and validation of the program as a basis for a joint dis- 
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sertation. The resultr of their research form the basis for an evaluation. 

The program consisted of two slide* carousels (192 slides) with accom- 
panying cassette tape and workbook, as well as three 8 mm cassettes with 
accompanying manual and monitor's guide. Program effectiveness was tested 
using a post-test only control group design, with 40 subjects assigned 
randomly to both experimental and control groups. The experimental group 
received the instruction prior to taking the post-test; the control group 
did not. The comparison was then made between the post-test performance 
of one group who had received instruction and a second group who had not. 
Subjects also completed a questionnaire about the program. 

The findings were of significant differences between the experimental 
group (x=218.15) and the control group (x=84.55) at the p<.001 ic;vel. In 
addition, a student feedback response form showed the multimedia approach 
as being ''highly interesting", of value to students, and an efficient way 
to present this material. 

The conclusion of effectiveness in instruction was drawn from these 
data. The question of relative effectiveness, this approach in comparison 
with another approach, was not asked. Thus, while one may conclude that 
this approach worKS , one has no information to conclude its superiority or 
inferiority to that of a more traditional approach (e.g. a lecture method 
or a programmed text). 

Business Administration 601/602: 
Quantitative Methods for MBA Students 

Based upon work done under a faculty development grant by Dr. Allen 
D. Karchner, a self-study program of quantitative methods was begun in 
winter quarter, 1974. Four subjects are available: linear algebra, calcu- 
lus, statistics, and operations research. The program was designed to 
appeal to three potential audiences (1) entering MBA students who had done 
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comparable undergraduate work and who wished to *'test out" of the require- 
ment; (2) students who wished to work independently and at their own rate; 
and (3) students who are off-campus and thus unable to attend classes with- 
out considerable difficulty. 

To date, no definitive stu^ly has been done to compare achievement scores 
of students using this method as compared to those students who take regular 
course work. However, considerable data are available to evaluate the pro- 
gram through indirect mea"s, such as completion rate and profile of non- 
completers. These findings give an indication of the type of student who 
can be expected to complete the program. 

After two years of operation of the program, on-carapus registration has 
averaged 12 students per quarter with an average of 19 credit hours granted. 
Average off-campus registration has been 9.6 students per quarter with an 
average 10.3 credits awarded. 

Of the approximately 600 exams which have been administered, 51.8 per 
cent have been graded ai the A level (88 or above), 33.5 per cent have been 
at the B level (77 to 87), and 14.7 percent have not been passing (below 77). 
The overall rate of completion of the course is 61.3 per cent of those who 
register for the course and take at least one exam. Students taking the 
course on campus have a greater probability of completing than those regis- 
tered off-campus through the various extension centers, 63.2 per cent as 
opposed to 58.1 per cent. This is understandable as there is less oppor- 
tunity for contact with the instructor and many students in this situation 
have not made the commitment that an on-campus student has. In fact, it 
would seem somewhat surprising that only 5 percentage points separate the 
two completion rates. Of those that complete, about two-thirds finish 
during the quarter of registration. 
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Some additional findings of interest are that (1) the percentage of 
repeat exams-- for the purpose of raising the grade--is greater for on campus 
students, 15,3% as opposed to 8.1% for those off-campus; (2) alTiost half 
(47.7 per cent) have registered for modules in more than one area during a 
quarter; and (3) about 7 percent of the exaiu^ taken so far occurred during 
quarter breaks, indicating an increase in flexibility of options. 

These data are of the type generated by a well -kept management 
information system, which Dr, Karchner has carefully maintained. The 
finding that many students do complete--68 to date--is counter-balanced 
by the finding that many do not, particularly if the substantial number 
who took no exams is included. When all students who have registered are 
considered, over half (55%) will not be expected to complete. Further 
study might determine the motivation or selection factors which operate 
in this case. 

Sociology 101: 
Introduction to Sociology 

Citing the problems of individual student contact with the instructor 
for a large survey class. Dr. Bradley Parlin proposed to redesign the lab 
sections of the introductory sociology course, using a modular approach. 
Original 1/, eight such modules were to be developed. As things turned 
out, three rDodules were eventually completed. These involved mainly adapta- 
tions of existing materials to produce slide-tape modules and a student work- 
book covering the three units. 

The evaluation of this effort consisted entirely of student opinion data, 
gathered through a questionnaire, toward the lab sections. Student opinion 
was solicited during spring 1975 regarding the previous lab instruction method 
These findings were contrasted with the revised method employed fall quar^-er. 
Then, suspecting that some of the observed differences were due to lab instruc 
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tor personalities, a final questionnaire was given, with the results analyzed 
separately lor each instructor. 

The following conclusions were drawn from the three questionnaires given: 

1. There were significant differences between instructors as perceived by 
students, as to their knowledge of the subject, their ability to respond 
to questions, and their personal involvement with students, 

2. The same kinds of differences in student attitudes were apparent in opinions 
toward the modules. However, regarding the overall lab e/perience, few 
differences of significant proportions were noted. 

3. Data suggested positive affect toward clarity of module objectives and 
examples in the text. There was reasonably positive feeling toward 
the usefulness of the modules as an addition to the course. 

4. In comparisons involving total class means, several instances of regres- 
sion toward the mean were noted for previously significant differences. 
For example, item number one dealt with the degree to which the lab in- 
structor came to know the students on a personal basis. Results from 
the previous two quarters had shown significant differences in favor 

of previoas method. However, the winter quarter mean for that item fell 
between the previous two means. Similar results were apparent for five 
other items. 

5. One item of particular concern was an item which asked students to rate 
this lab experience in comparison with previous lab experiences. Stu- 
dents consistently rated the previous method significantly higher than 
the revised approach. 

6. Where the fall and spring comparisons showed various items higher than 
others for each approach, the winter comparisons were uniformly signi- 
ficantly lower than either the fall or the spring comparisons. 

From the standpoint of the ID office, the chari^es brought about were 
relatively rainor--three of the eight original modules--and these were of ques- 
tionable merit as judged by student feedback. No comparisons of achievement 
under the two methods were attempted, as the test changed considerably from 
quarter to quarter. The contact between the ID office and the professor 
wa minimal in this case, and, judging from the available data, the changes 
in lab portion were not major. The considerable data available do not 
demonstrate major impact from this grant. 
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S econdary Rdurntion 610: 
Improvement of Reading in Secondary Schools 

Recogni7.ing that a major difficulty faced by high school students in- 
volves inadequate reading ski 11.'.^, this course was originally developed to 
train secondary teachers, regardless of subject matter expertise, in tech- 
niques of teaching reading. The course was made a requirement for all 
secondary education majors. 

Under a faculty development grant to Dr. William Strong, this course 
was revised extensively. Lila Phillips, a doctoral candidate in the depart- 
ment, worked to develop two modules which could be incorporated into the 
existing curriculum. These modules were meant to provide a self-directed 
alternative to the regular lecture course. Each module contained the 
following learning activities: rationale, objective, pre-assessment , 
learning alternatives, post-assessment, and remediation — all designed 
to help the student acquire and demonstrate a set of competencies. To- 
gether, the modules require roughly half of the quarter for completion. 

Prior to designing the modules, two research studies for assessing 
needs were conducted. The first was a survey of 4,000 Utah high school 
and junior high school students to identify the books read most frequently. 
A major finding of the study was a substantial drop in independent reading 
for high school students, in comparison with junior high students in the 
same geographic areas. For whatever reason, these students were reading 
considerably less on an independent basis during the high school years. 
The second study convened groups of secondary teachers-. State Department 
of Education personnel and USU faculty members to identify priority areas 
for emphasis in teaching reading improvement skills to secondary teachers. 
The findings of this study were incorporated into the course redevelopment 
as the main areas of emphasis for the modules. 
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Two modules were developed for use in the required class, one dealing 
with study skills and a second dealing with reading comprehension skills. 
To test the effectiveness of these modules, during spring quarter 1976 a 
"pre-test - post-test control group design*' (Campbell and Stanley, 1963) 
was employed. One group of 47 students took the class with the new modules 
while a second group of the same size took the class using the previous 
lecture method. Each group received a pre- test based upon the objectives 
of the module and then later, following the instruction by lecture or module, 
received an alternative form of the test covering the same objectives but 
at a greater level of difficulty. 

The results of an analysis of covariance statistical test for the first 
mO'Vile are shown in Table 3. In this comparison, the results are not signi- 



Pre-test 
Post-test 



TABLE 3: RESULTS OF ANCOVA 
MODULE I: STUDY SKILLS 

Experimental 
Group (n=47) 
Mean Scores 

21.42 
90.10 



Control Group 

(n=47) 
Mean Scores 

18.95 
90.74 



Possible score, each test: 100 
F = . 126, df 1, 91 

Differences not significantly different 



ficant, indicating no clear superiority for either approach. This was inter- 
preted as indicating that the new approach taught as well as the previous 
approach. 

The results of .e second module, shown in Table 4, indicate clear 
superiority for the traditional approach. Here the results are signifi- 
cantly different in favor of the previous method. While both methods were 
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TABLE 4 : 



RESULTS OF ANCOVA FOR MODULE II: 



COMPREHENSION SKILLS 



Experimental Group 
(n=47) 



Control Group 
(n=47) 



Pre-test 



28.34 



14.34 



Pn lit- test 



88.36 



92.57 



Possible score, each test: 100 

F = 5.453; dfl, 91 (F*=4.0, p<.05) 

Differenced significantly different in 
favor of the control group 



effective in attaining the established criterion levels, clearly, this module 
needs revision before implementing it on a regular basis in the classroom. 
Some few clues to likely areas for improvement come from a criteria check- 
list filled out by those using the module approach. Students noted a lack 
of opportunities for equivalent practice of the needed skills. Somewhat 
ironically, the student ratings according to specified criteria were higher 
for the second than for the first module. Some additional formative evalua- 
tion may be needed to make the necessary revisions. 



Dr. Larzette Hale of the U.S.U. Accounting Department received two 
grants from our Division, A mini grant enabled her to conduct a survey 
study of integration of current professional accounting standards and in- 
formation into selected undergraduate accounting courses. A subsequent 
faculty development grant enabled her to completely redesign her Accounting 
Theory course. The following is a statement of her basic thesis and a de- 
scription of the survey she conducted: 



Accounting 561 : 
Accounting Theory 
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The professional practice of accounting in the United 
States requires strict adherence to accounting and auditing 
standards. These standards include the most recent pronounce- 
ments of all appropriate boards and commissions. Furthermore, 
these same current pronouncements and info^rmation are included 
on the CPA Examinations within a year after issuance. There- 
fore, accounting students need to be aware of, and understand, 
the current literature in their field. To teach to this need, 
accounting educators must deliberately plan for and integrate 
the most current materials into the day-by-day teaching-learning 
experiences , 

When a survey of literature in the accounting field failed to yield any 
information on the present state of this problem, Dr, Hale began her study 
of the integration of current professional accounting standards and infor- 
mation into s ected undergraduate accounting courses. Beginning with 
a mini grant, tools were developed to assess the updating of accounting 
education at the state and local (U,S,U,) level. The first was a question- 
naire assessing how teachers in three Utah universities update accounting 
courses and how they evaluate their efforts, 

A second tool was a student questionnaire designed for local use at 
U.S,U, Part I of the questionnaire assessed student understanding of eight- 
teen general education items. Dr, Hale feels about 2/3 of the students 
should know about two thirds or 12 of these items before they enter the 
Accounting Theory course which is based almost exclusively on current 
material. Her goal is to bring the department faculty awareness to the 
level at which students entering Theory will know about 3/4 or about 14 
of these items. So far the questionnaire has been administered at the 
beginning of the Accounting Theory course three times. Evidence thus far 
indicates a growing awareness on the part of students of current pronounce- 
ments in the accounting field, 

A direct result of the minigrant study was a nationwide study of the 
same research question, namely, to what extent are current accounting 
standards, pronouncements and information integrated into undergraduate 
accounting courses. Her major finding was that an overwhelming majority 
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(94%) of accounting educators feel that such information should be intergrated 
into courses to which it is applicable. By contrast, only 13.6% of those 
sampled believed that a separate course dealing specifically with current 
accounting literature could be justified at the undergraduate level (Hale, 
1976). 

As an additional outgrowth of the original mini grant, Dr. Hale revised 
an accounting theory course. Accounting 561, to reflect the current litera- 
ture in the field. Integrating these concepts into her course was done 
extensively and included the production of appropriate slide-tape materials. 
The course was taught during Winter 1976 with the course partially revised 
and again in Summer 1976 with the finished product. A comparison of final 
exam scores was made using analysis of covariance to equate the groups on 
the basis o^ end of quarter overall grade point average. The results, shown 
in Table 5 below, show significant differences (F=ll .57 ; df=l,44) between the 
two groups in favor of the fully completed products. A comparison between 



TABLE S 

FINAL EXAM GRADE POINTS FOR ACCOUNTING 561 COURSE, 
USING GPA AS A COVARIATE 



Winter Summer 

1976 1976 

N 27 20 

Mean GPA 3.08 3.24 

Mean Final Exam Score 184.48 210.35** 

Adjusted Final Exam Score 185.90 208.52 



F= 11.57 df= 1,44 F**= 7.24, p<.01 

**NOTE: This score was adjusted due to the shorter nature of summer 
quarter and a somewhat abreviated versioii of the test used, 
with 200 possible points in Summer compared with 250 possible 
points on the Winter exam. 



the revised method and the original version of the course was not possible 
because of the completely different content covered in the examinations. 
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In summaiy, the original minigrant for survey research led to a fac- 
ulty development grant to revise the Accounting Theory course. Four specific 
outcomes were as follows: (1) the original survey results were distributed 
throughout the state; (2) a second study, funded and conducted nationally, 
led to an article for publication in a professional joumel; (3) other pro- 
fessors in the department are beginning to incorporate current information 
into their ^^.ourses as evidenced by higher awareness levels of entering 
students; and (4) the general accounting theory course has been revised 
and has been shown to be superior in its final version to the interim 
version of the course. 
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IV. RESULTS OF DEPARTOENTAL EFFORTS 
IN CIVIL ENGINEERING AND WILDLIFE SCIENCE 

The departmental stage of the U.S.U. ID model actually extends the 
faculty development stage of our model to involve consulting entire depart- 
ments on curriculum redevelopment. The basis for instructional development 
at a departmental level is a thorough needs assessment, resulting in a priori- 
tized set of needs for course development. We are now using a model de- 
veloped and evaluated by a member of our staff (Eastmond, 1975 a and b) . 
The needs assessment process overcomes some of the problems inherent in 
Stage IV by allowing the developers to work within the goals of the de- 
partment. The consultant acts as a process person who attempts to task- 
orient the redevelopment effort. In this stage, not only can more resources 
be focused on a problem, but a more systematic approach to a solution is 
possible because more affected subsystems are open to be investigated 
and revised. 

The two departments involved to date have been Civil Engineering and 
Wildlife Science, the former beginning a year prior to the latter. Included 
with the results for Civil Engineering are those for Fluid Mechanics, one 
of the original faculty development grants given by the ID office, and a 
'^spinoff*' into Mechanical Engineering which was an outgrowth of the Civil 
Engineering effort. Only one course is available for reporting from 
Wildlife Science, although others are being developed and evaluated at this 
time . 

Civil Engineering 350 : 
Fluid Mechanics 

One of the earliest and most dramatic evaluations of a course was 
that done in 1973 by Cheryl Whitlaw, a graduate student, comparing two 
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approaches to teaching the Fluid Mechanics course. Dr. Gordon Flajiuner 
had developed a self-paced approach to teaching the course under a 
released-time grant from the Instructional Development Division. 'ITiis 
evaluation, conducted by the professor and graduate student, compared 
student achievc'iient and attitudes under the self-paced approach vs. the 
lecture method of teaching the class. 

The design was essentially a pretest-posttest control group design 
(Campbell and Stanley, 1963), except that assignment to groups was not done 
on a completely random basis. Two sections of the course were taught. A 
class beginning at 7:30 A.M. three times per week was given the self-paced 
version of the course. A second class taught at 10:30 A.M. employed the 
traditional lecture method. To insure the equivalence of the two groups 
for comparison, both a pretest of achievement and a measure of motivation 
were given and analyzed separately for the two classes. The conclusion, 
based on the observed similarity of the two groups, was that they were 
statistically comparable. 

Fol'lftWing course completion, two final examinations were given to all 
students, one open-book focusing on problem solving skills and one closed- 
book focusing mainly upon concept learning. Two versions of each test 
were assigned randomly to students. In addition, student were asked to 
report time spent in doing work for the course. 

Results 

The result of the final exams for the two groups are shown in Table 6. 
The findings for the openbook section showed no significant differences 
between the self-paced and lecture portions of the course. However, for 
the closed book test, there were major differences between the results 
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TABLR 6 : 



COMPARISON OF MEAN SCORES FOR THE CE 350 FINAL EXAMINATIONS 



Lecture Section 

(N=24) 




Open Book Test 



Form 1 
Form 2 



80 
74 



74 
78 



-1.32 
0.78 



Closed Book Test 



Form 1 
Form 2 



46 
55 



81 

77 



8.56 
3.09 



* Significant at P<.01 level 

of the two groups for the closed book test. This finding was interpreted 
to mean that students learned problem-solving skills equally well under 
both systems, but learned the essential concepts of the course better under 
the self-paced method. 

An additional finding is that students report spending more time 
under the self-paced than under the lecture method. Students in the self- 
paced class report spending an average of 123 hours, as opposed to 114 
hours for the lecture class students. 



Some concern for the comparability of the two groups is in order. It 
is likely that a different type of student will sign up for a 7:3.0 class 
than will be found in one during mid-morning. Of course, meeting the 
schedules imposed by other courses will act as an additional factor. 

An additional concern was that students under the self-paced method 
took the final when they completed the work, some as much as one month 
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after the quarter's end. The possihLlity of collaboration with classmates, 
even given the use of alternative test forms, is a very real concern. 

Manufacturing Engineering 4i5, 416 
M aterials Science 

Revisions to this cojjrse, taught by Professor Karl Somers, centered 
around a series of 25 slide-tape units. These self- instructional units were 
made available to students during lab time and after hours through A-V Services 
in the Library. The major benefits of this approach have been (1) freeing 
up instructor time to work with individual students; and (2) clarifying the 
demonstrations of complex equipment operation. 

The evaluation included measures of both student achievement and student 
attitudes. Achievement was measured at three points in time: (1) prior to 
the course revision, Spring 1975; (2) during Fall 1975, when modules were 
complete with a temporary sound track; and (3) during Spring 1976, when the 
course was essentially complete. 

Achievement . The class in Manufacturing Engineering 462 is taught twice 
per year, during spring and fall quarters. Because of the lack of experimental 
control over students enrolling in the class, "a non-equivalent control group 
design** (Campbell and Stanley, 1963, pp. 47-50) was used. Scores on the final 
test under the previous method of teaching--lecture/demonstration--were com- 
pared with alternative form test scores under two quarters of the new approach. 
The groups were equated statistically using analysis of covariance (ANCOVA) 
with grade point average (GPA) as the covariate. Comparisons of the three 
quarters* scores showed significant differences CF=2.54, df=2,41, p.<.10) in 
favor of the two experimental groups. The results are summarized in Table 7. 
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TABLE 7: MEAN SCORES FOR 1HREE GROUPS OF ME 462 STUDENTS 

Number of Mean Mean Score Standard Deviation 

Students G.P.A. on Final On Final 

Spring Quarter 1975 

Previous method 14 2.7.3 72.93 10.34 

Fall Quarter 1975 

Previous Method 21 2.86 80.10 9.99 

Spring Quarter 1976 

Experimental Method 10 3.14 83.80 8.61 



From the findings, some trends in achievement may be appearing. One 
of these is that the mean G.P.A. of students enrolled seems to be rising. 
If ''grade inflation" can be discounted, it would appear that students of 
high academic ability are enrolling in the class. The mean score on the 
final has gone up considerably, and even with an adjustment for higher 
entering G.P.A., students under the new method are scoring significantly 
higher than under the previous method. The decreasing standard deviation 
indicates a fairly sizeable reduction in the spread of scores, i.e. stu- 
dents are scoring more consistently, with less variation. While further 
research could verify these apparent trends, each of these findings would 
argue for superiority of the new method over the old one. 

Attitudes . Two survey instruments were used to assess student atti- 
tudes toward the course, both given during fall quarter 1975. While the 
major purpose of the survey was for student feedback during course develop- 
ment, a number of the questions can be applied in a sununative sense to 
document project accomplishment. 

For example. Table 8 shows seven statements with high agreement in a 
favorable direction among students. 
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TABlii 8 : RESULTS OF 'nil- MI: 462 ATTrPUDI: SURVI-Y, FALL QUARTER 1975 

(n=20) 

Percent of N 
Answering Favorably* 

1. I think the worksheet questions in the lab 

manual were generally clear 85 

3. I like being able to repeat a slide/tape 

module by myself if I need to, 90 

6. Subject material was organized in a way 

that seemed logical to me, 85 

13. I found that the factual information 
given in the slide/tape modules was 

both accurate and useful in the lab. 95 

14. I felt I had enough personal contact 

with the instructor. 85 

15. I felt that the lab experience was 
necessary to completely understand 

the course. 95 

20. Important ideas were easily recognized 

in most slide/tape modules. 80 

*Favorable responses would include '^Strongly Agree*' or ''Moderately Agree" 
for a positive statement, and "Strongly Disagree" or "Moderately Disagree" 
for a negative statement. 



Students apparently liked the factual content of the course, its clarity and 
organization, as well as the close contact with the instructor. 

Concerning responses unfavorable to the course, on only one item (#7) 
did more than 25 percent answer unfavorably. This item stated, ''Some por- 
tions of the slide/tape presentations raised questions which were not an- 
swered in the lab," and unfavorable response was interpreted as agreement 
(70%) with the statement. This particular item could also be viewed as 
stimulating curiosity and thus may not be particularly unfavorable to 
the aims of the course. At any rate, during ision particular attention 
was paid to clarifying unanswered questions. 
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It should be noted that these :;r!'on};Iy positive findings were obtained 
concerning the field test versioi\ of the presentation, before the professional 
sound script. While further ineasurLng of student affect has not been attempted, 
it would likely bo as favorable or more favorable toward the revised, com- 
pleted slide/tape modules. 

Civil Hngineering 430 and 531 
Soil Mechanics 

A revision of the Soil Mechanics sequence of courses taught to senior 
Civil Engineering students required the combined efforts of three professors: 
Drs. Anu -rson, Dunn and Kiefer. The medium chosen was essentially a textbook 
format, supplemented with overhead transparencies and realia as appropriate. 
The revised course was taught for the first time during fall quarter 1975. 
Indicators used for the evaluation include (1) instructor's perceptions; 
(2) questionnaire results; and (3) enrollment data. 

The perceptions of the course instructor, Dr. Irving Dunn, suggest 
that certain changes in the course were discemable. One was instructor 
enthusiasm, which Dr. Dunn reports runs in 5 year cycles, with each course 
revision. His own enthusiasm was high for the new approach. After examining 
tests from previous years. Dr. Dunn concluded that the present grades are 
higher, but he questions exactly what this means. With the change in tests 
and the change in patterns of grading, it was virtually impossible to make 
a valid comparison, he felt. The quality of student discussions had remained 
the same, he believed, with insightful comments from the capable students 
and the regular complaints from the regular complainers, the less able stu- 
dents. 

Enrollment data shown in Table 9 indicate a doubling of the percentage 
of students who continue in the Soil Mechanics sequence beyond the required 
course since the introduction of the new approach. The percentage of students 
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continuing in the oi)ti<)nal CDurst* incrcasoil from an avcra>»o of 17 porccMU 
in the provLous three years to 83 percent tluring 107(i. 



TAfU.fi 

STUOfiNT fiNKOUMiNT AND PfiKCliNT CONTINUING IN 
SLiNIOK iJiVfil. son. MiCIIANICS SnqUENCIi 



Year 



linrollmcnt in CE 531 
(required) 

Enrollment in CE 532 

(required for some, optional 
for others) 

Percent Continuing 



li)73 
38 

17 
45 



1974 
34 

18 
53 



35 

15 
43 



1976 
24 

20 
83* 



*The 1976 school year was the first in which the revised materials were 
used. 



A student questionnaire given at the completion of the fall 1975 
course gave some indication of student sentiment toward the course. Students 
responded favorably toward the style of teaching, the logic and illustrations 
in the text, and the use of visual materials in the presentation. Formative 
feedback from the same questionnaire suggested some areas for improvement 
in future course revision in the areas of providing sufficient examples, 
maintaining the logic of the argument, and presenting excessive information 
on the transparencies. Revision of the course has been conducted over a 
one year period, and the new format is presently receiving its second trial 
run. 

Wildlife Science 390: 
Game Birds and Mammals 

The Wildlife Science Department became the second thrust for ID efforts. 

Following an extensive assessment of educational needs (Eastmond 1975), the 
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undergraduate course "WLS 390: Game Birds and Mammals" was selected for 
revision. Rather than granting release time to a faculty member, a post- 
doctoral fellow. Dr. Fritz Knopf, contracted to act as an intermediary be- 
tween the ID Center and the departmental professor. Dr. Michael Wolfe. The 
major development activities were handled by Dr. Knopf and an ID graduate 
student; Dr. Wolfe and ID staff members were brought in as needed. An in- 
dividualized instruction program was developed for the portion of the course 
dealing with upland and other game birds. The instructional package, con- 
sisting of a 33 page booklet was designed to be used with an accompanying 
textbook by P. A . Johnsgard, North American Game Birds of Upland and Shoreline. 
For each unit, a rationale statement, a performance objective, and a worksheet 
were presented in the instruction package; for detailed information needed 
to complete the work, the student referred to the text by Johnsgard. Written 
material was supplemented by color slides available in the Audio Visual 
Center of the Merrill Library. 

During winter quarter, 1976, as a pilot test of the new approach, stu- 
dents in WLS 390 received a lecture presentation from John Ratti, Teaching 
Assistant, for the first three weeks of the course. Following the first mid- 
term, students were presented with the experimental instruction packets, 
instructed in their use, and then given the next three weeks, on their own , 
to complete them. During that time period no formal classes were held. At 
the end of the three weeks, a mid- term examination was given covering material 
taken directly from the modules, along with a student opinion questionnaire. 
The remainder of the course was taught in similar fashion to the first three 
weeks, using lecture presentation. A similar modular approach is being 
developed for this portion of the course. 
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Methods of Comparison 

This evaluation compared the results of the questionnaire and mid- 
terra examinations. In doing so, the following considerations must be kept 
in mind: (1) the questionnaire was administered only once, with students 
asked to compare their experiences under the two approaches; (2) the two 
mid-term examinations covered different material, and while judged by the 
course instructors to be of similar difficulty, they represented tests of 
different content areas. 

Some useful conclusions can be drawn from the data available. 

Results 

The form of comparison for the mid-term exams used the experimental 
subjects as their own controls, i.e. the same subjects were used under both 
experimental and control conditions. Table 10 summarizes the results of the 
two mid- term exams. Both had a possible 100 point maximum. 



TABLE 10: 

A COMPARISON OF MID-TERM TEST MEANS TAUGHT UNDER 
TOO METHODS OF INSTRUCTION 

Lecture Individualized 
Presentation Present at ion 



Number of students (n) 52 
Mean Test scores (x) 86.7 
Standard Deviation 11.5 



49 

93.9 
8.0 



Given the equal difficulty of tests, the data suggest that the new method 
of presentation is at least as satisfactory as the previous lecture method. 
The differences between the correlated means are significant t=4.86, df=47, 
p<.05) in favor of the experimental presentation. 
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Items from the student critique form were scored separately on a 
seven point scale, from strongly agree to strongly disagree, with 4 as a 
"neutral** response position. No attempt was made to compute a score com- 
bining all items. Where items were phrased unfavorably toward the program, 
scoring was done in the opposite direction (shown in parentheses for those 
items). Thus, the higher the score bn a particular item, the more favor- 
able the student opinion expressed. 

The following item means fall at least one point away from the neutral 

position and can be taken as expressive of favorable opinion toward the 

class: Item 

Mean 

1. The performance objectives written in the instructional 

packages are generally clear. 6.48 

2. There were times when I was not clear about what was 
expected from the unit. (Scored in the opposite 

direction). 5.65 

9. Ifliile recognizing that the technical quality of a 
number of slides could be improved, I do feel that 

the important information could be obtained from them. 5.21 

11. I felt that the blue module booklet was a significant 

addition to the course. 5.75 

12. The page layout in the taxonomy sections was helpful for 
memorizing. 5.54 

13. I thought the test questions for the three-week package 

covered material I had seen before. 6.04 

These would seem to be encouraging findings. Students feel that the 
objectives are clear, that the slides and the module booklet are signi- 
ficant improvements in the course, that the page layout aids in memorizing, 
and that the test questions covered previously learned material. 

Some clues to additional areas for improvement were given in the items 
where student opinion was less favorable toward the present course revision. 
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The following items should be viewed in this manner. 

4. Making this course 3 credit hours would be a step in the 

right direction. 5.29 

8. In the future, we anticipate having either a professor or 
T.A. available one hour weekly for answering any questions 
you may have. Concerning this arrangement, 

a. I think I would probably take advantage of this weekly 
contact if it were made optional to students 5.48 

b. I think this type of arrangement would improve this 

aspect of the course by adding the "human element''. 5.40 

14. Some questions requiring a bit more interpretation and use 

of the knowledge gained should be included. 6.08 

Students favor having this course carry three credit hours. They would 
like to have an instructor available to answer questions, and they would 
prefer more test items involving integration of the knowledge obtained. 
To the suggestion of adding a fish section to the course or having a sep- 
arate fish course, the responses were fairly ambivalent. 

In conclusion, the course revision to this point has been perceived 
favorably by students, and there have been no indications that learning 
suffered. Certain improvements, involving raising the number of credit 
hours given, increasing student-instructor contact, and requiring more 
integration of knowledge on the test questions are viewed favorably by 
the students. 
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CONCLUSIONS 

The complexities of evaluating projects as diverse as these are some- 
times overwhelming. Reported herein are a series of projects where evidence 
regarding student achievement or student attitudes was available, over a time 
span of some four years. The sample of rainigrants is necessarily small-- 
three out of more than one hundred given- -because the recipients of small 
grants are neither asked nor expected to produce such data. The minigrants 
included are fairly atypical of others conducted, in each case being con- 
siderably more extensive. The faculty development grants described cover 
roughly one third of the total given--nine out of twenty-seven completed to 
date. While a more complete coverage of all projects with extensive evalua- 
tion would have been desirable, such an approach has not been feasible given 
the manpower available, and in many cases runs counter to the prevailing 
philosophy of this center. To be effective, evaluation must be something 
planned jointly with the faculty client and carried out to meet his specific 
needs, tfhen the client balks or views this activity as an imposition, the 
evaluator is obligated to back off and take a new approach. If no satisfac- 
tory new approach can be found, the evaluation must simply take a lower 
priority position on the agenda. In no case will a client be hounded 
simply to provide data for the ID office purposes. The client's good feel- 
ings about his own project, and the good will of both parties, are simply 
of greater importance. 

The finding most significant from this survey of projects is that the 
redesign and development of courses can work, that this process has been 
demonstrated effective to a greater or lesser extent for a number of projects 
undertaken by this office. The variety of these projects is considerable, 
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but the finding~-with some few except ions--is that student achievement can 
be boosted or student attitudes toward the material can be improved or both 
through the use of faculty initiated instructional development. While 
none of the reported experimental designs are flawless and few of the 
projects show gains as striking as an evaluator would prefer, there is 
still a solid body of findings suggesting that modest attempts at involving 
professors in the redesign of their own courses can have substantial im- 
pact upon students. And that is what instructional development is all about. 
The results reported herein, multiplied by many students over a number of 
quarters, has a Potentially sizeable impact. So there is something different, 
and that difference can be measured in students. 
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